A Framework for Testing Properties of Discrete Distributions: Monotonicity, Independence, and More

Given data sampled from an unknown discrete probability distribution p, does the underlying distribution possess some property of interest? For instance, is the distribution uniform? Monotone? Are its marginals independent? This class of problems is one of the most fundamental questions in statistics, where it is known as hypothesis testing. Classical work on this problem has focused on distributions over a domain of a fixed size, as the number of samples goes to infinity. However, in modern scenarios, distributions may be over massive domains, and we are often limited by the number of samples or computational power. As such, over the past two decades, there has been intense study with these goals in mind (see [2] for a survey). Nevertheless, even for many basic properties of distributions, the optimal sample complexity was unknown. We provide a testing framework which achieves the optimal sample complexity for the properties mentioned above, and more. Notably, all properties we study have strongly sublinear complexities, requiring only a number of samples proportional to the square root of the domain size. The framework follows a conceptually simple learn-then-test approach. Naively, such methods seemed to be intrinsically statistically inefficient, due to an information-theoretic lower bound for robust `1 identity testing [3]. We bypass this lower bound by using χ2 distance as an intermediary metric. χ2 is a non-uniform rescaling of `2, and is more “punishing” than `1. This makes learning in χ2 slightly harder, but testing in χ2 drastically easier. This work appeared at NIPS’15 as [1]. BODY Wanna optimally test if your distribution has a property? Assume it does, learn the distribution, and test the hypothesis. Use χ2 distance!